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Abstract 

Background: Recent terrorist attacks and natural disasters have led to an increased awareness of the importance of 
emergency planning. However, the extent to which emergency planners can access or use evidence remains 
unclear. The aim of this study was to identify, analyse and assess the location, source and quality of emergency 
planning publications in the academic and UK grey literature. 

Methods: We conducted a scoping review, using as data sources for academic literature Embase, Medline, Medline 
in Process, Psychinfo, Biosis, Science Citation Index, Cinahl, Cochrane library and Clinicaltrials.gov. For grey literature 
identification we used databases at the Health Protection Agency, NHS Evidence, British Association of Immediate 
Care Schemes, Emergency Planning College and the Health and Safety Executive, and the websites of UK 
Department of Health Emergency Planning Division and UK Resilience. 

Aggregative synthesis was used to analyse papers and documents against a framework based on a modified FEMA 
Emergency Planning cycle. 

Results: Of 2736 titles identified from the academic literature, 1603 were relevant. 45% were from North America, 
27% were commentaries or editorials and 22% were event reports. 

Of 192 documents from the grey literature, 97 were relevant. 76% of these were event reports. 

The majority of documents addressed emergency planning and response. Very few documents related to hazard 

analysis, mitigation or capability assessment. 

Conclusions: Although a large body of literature exists, its validity and generalisability is unclear There is little 
evidence that this potential evidence base has been exploited through synthesis to inform policy and practice. 
The type and structure of evidence that would be of most value of emergency planners and policymakers has yet 
to be identified. 
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Background 

An effective and efficient emergency response can re- 
duce avoidable mortality and morbidity after a mass cas- 
ualty incident. Numberous guidance documents have 
been issued since the Civil Contingencies Act placed a 
legal responsibility for emergency planning with NHS 
organisations [1-3]. Following the 7 th July 2005 London 
bombings Lady Justice Hallett's Rule 43 report raised 
concerns about the capacity of the London Ambulance 
service to plan communications for a major incident, the 
ability to establish a Gold Control Room and the process 
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of triage [4]. Similar comments about the mismatch be- 
tween plans and events followed the 9/11 World Trade 
Center attacks [5]. 

The process for developing policy and guidance docu- 
ments is variable with the strength and quality of the 
underlying evidence base often proving unclear. For ex- 
ample the UK Emergency Preparedness document gives 
no information on the provenance of its evidence base 
[6]. The review team was commissioned by the UK Na- 
tional Institute of Health Research to conduct a scoping 
review of the academic and UK grey literature to identify 
the location, source and quality of emergency planning 
publications. This would allow us to characterise the evi- 
dence that exists to inform emergency planning. 
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Methods 

Academic literature identification 

Following compilation by the authors of themes and 
topics considered relevant to the field, pilot searching 
was carried out in the subarea of health services busi- 
ness continuity. This is an accepted stage for all scoping 
reviews as it seeks to establish an optimal balance be- 
tween sensitivity and specificity. This is important con- 
ceptually, to finalise the scope of the review, and 
pragmatically, to ensure that the review is feasible within 
the available time and resources [7]. A final search strat- 
egy was then developed to retrieve evidence relevant 
to the whole subject area, subdivided into Business con- 
tinuity, Hazard analysis, Capability assessment and main- 
tenance, Recovery, Communications/informatics and 
Organisational behaviour/Human Resources. This final 
search aimed to identify slices of the evidence, but was 
designed to be more focussed, producing a higher yield 
of relevant papers and was therefore more time effect- 
ive to review. The information specialist searched the 
electronic databases Embase, Medline, Medline in Pro- 
cess and Psychinfo (via Ovid SP), Biosis and Science 
Citation Index (via Web of Science), Cinahl (via EBSCO), 
the Cochrane library (via Wiley) and Clinicaltrials.gov 
(Table 1). Searches were conducted in November 2010 
with searches covering the period January 1990 to Octo- 
ber 2010. We did not limit to "human" as we wished to 
identify literature on bioterrorism and zoonoses that 
might be relevant. 

Extracted titles and abstracts were screened by KC, 
AL, PG and AB. Each title or abstract was reviewed and 
deemed to be relevant, equivocal (subject matter sug- 
gestive of relevance to emergency planning and/or 
management), not relevant or containing inadequate in- 
formation for coding (limited PRISMA chart figure 1). 
To operationalise the brief from the commissioners of 
the study to prioritise research relevant to UK health 
emergency planning we sought to include literature relat- 
ing to comparable health services. We therefore excluded 
articles relating to non-health emergency planning, non- 
emergency planning, and non-UK legislative issues, and 
those from low- and middle-income countries unless 
they were likely to be generalisable to the UK. Two sub- 
sets of two hundred references were coded by a pair of 
researchers (either KC/PG or ALIAS) and kappa values 
for agreement calculated to examine consistency. 

Grey literature identification 

We aimed to identify grey literature that included primary 
data or analysis that could be used to inform decision- 
making. Potential sources of grey literature were identified 
in advance by our expert advisory group and supplemen- 
ted by the academic literature review and concurrent 
stakeholder interviews (reported elsewhere). Based on this, 



Table 1 Search strategies 



Business continuity 1. Disasters/pc 



2. (emergency response or emergency preparedness 
or emergency plan$ or emergency operation plan$ 
or disaster or major incidents or incident plan$).ti,ab. 

3. 1 or 2 

4. (business continuity or organisational resilience 
or business interruption or adaptive capacity 

or strategic or coordination).ti,ab. 



Hazard analysis 



5. 3 and 4 

6. limit 5 to yr = "1? 
1. Disasters/pc 



-Current" 



2. (emergency response or emergency 
preparedness or emergency plan$ or emergency 
operation plan$ or disaster or major incidents 

or incident plan$).ti,ab. 

3. 1 or 2 

4. (hazard analysis or risk factor or risk assessment 
or forecasting simulation or modelling).ti,ab. 



Capability assessment 
or maintenance 



5. 3 and 4 

6. limit 5 to yr = 
Disasters/pc 



-Current" 



Recovery 



Communications/ 
informatics 



2. (emergency response or emergency 
preparedness or emergency plan$ or emergency 
operation planS or disaster or major incidents 

or incident plan$).ti,ab. 

3. 1 or 2 

4. (capability assessment or capability 
maintenance or gap analysis or needs 
assessment or drill or simulation or 
preparedness training).ti,ab. 

5. 3 and 4 

6. limit 5 to yr = "1990 -Current" 

1. Disasters/pc 

2. (emergency response or emergency 
preparedness or emergency plan$ or emergency 
operation planS or disaster or major incidents 

or incident plan$).ti,ab. 

3. 1 or 2 

4. (significant event analysis or serious untoward 
incidents or root cause analysis or debrief or 
organi?ational learning or rehabilitation).ti,ab. 

5. 3 and 4 

6. limit 5 to yr = "1 990 -Current" 

1. Disasters/pc 

2. (emergency response or emergency 
preparedness or emergency planS or emergency 
operation planS or disaster or major incidents 

or incident plan$).ti,ab. 

3. 1 or 2 

4. (communications or mass media or public 
relations or information systemS or 
information service$).ti,ab. 

5. 3 and 4 

6. limit 5 to yr = "1 990 -Current" 
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Table 1 Search strategies (Continued) 



Organisational 


1 . Disasters/pc 


behaviour 


2. (emergency response or emergency 






preparedness or emergency plan$ or emergency 




operation plan$ or disaster or major incidents 




or incident plan$).ti,ab. 




3. 1 or 2 




4. (community engagement or community 




involvement or participatory involvement or 




participatory engagement or consumer 




participation or organi?ational behavio?r 




or health personnel or human resources).ti,ab. 




5. *"Attitude of Health Personnel"/ 




6. interprofessional Relations/ 




7. 4 or 5 or 6 




8. 3 and 7 




9. limit 8 to yr = "1990 -Current" 



we searched databases at the Health Protection Agency, 
NHS Evidence, British Association of Immediate Care 
Schemes, Emergency Planning College and the Health 
and Safety Executive, and the websites of UK Department 
of Health Emergency Planning Division and UK Resili- 
ence. We also conducted an internet search using Google 
and Internet Archive for publicly accessible reports (public 
enquiries and coroner's reports) into the following previ- 
ously identified major incidents: 

Aberfan slag heap slip 
Hillsborough stadium disaster 
Ibrox stadium incident 
July 7 bombings 
Kegworth (Ml) aircraft crash 
Ladbroke Grove rail 



Lockerbie bombing 
Manchester airport take-off crash 
Marchioness pleasure boat sinking 
Piper Alpha oil rig explosion 
Potters Bar rail 
Summerland fire disaster 
Ufton Nervet rail crash 
Zeebrugge ferry sinking 

Literature analysis and synthesis 

Where the title and abstract were considered to be rele- 
vant or equivocal we extracted further information relat- 
ing to country of origin, type of publication and type of 
event discussed. Publications were coded on an Microsoft 
Excel spreadsheet using aggregative synthesis, a technique 
appropriate for exploring qualitative data where the con- 
cepts are secure, predefined and not contested[8]. We 
used a thematic framework developed a priori from the 
FEMA Emergency Management Cycle (Figure 2) [9]. This 
framework is widely known and used within the emer- 
gency planning community, and covers the range of issues 
in the field using mutually exclusive concepts. Framework 
synthesis can be used in conjunction with either aggrega- 
tive or interpretive reviews, has been shown to facilitate 
more rapid coding of the literature[10] and is, therefore, 
particularly suited to the objectives of a scoping review. 

Results 

Academic literature review 

We reviewed 2940 titles and abstracts. Of these, 625 
were coded by a pair of reviewers and with kappa values 
of 0.578 (AL/AB) and 0.740 (KC/PG). 1545 publications 
were felt to be relevant or equivocal. The country, type 
of publication and type of event are shown in Tables 2 



Records identified through 
database searching 
(n = 5966) 



Figure 1 Limited PRISMA flow diagram. 



Additional records identified 
through other sources 
(n = 204) 



Records after duplicates removed 
(n = 2940) 



Records screened 
(n= 2940) 



Records excluded 
(n = 1395) 



Abstracts analysed 
(n = 1545) 
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EMERGENCY 
PLANNING 



CAPABILITY 
MAINTENANCE 



EMERGENCY 
RESPONSE 






RECOVERY 



INFORMATICS & 
INTELLIGENCE 



COMMUNICATIONS & 
MASS MEDIA 



Figure 2 Adapted Emergency Management Cycle. 



and 3. For comparison, the number of events reported 
to the EM-DAT database at the Centre for Research on 
the Epidemiology of Disasters from 1990-2011 is also 
shown[ll]. EM-DAT is a multi-source validated data- 
base of disasters where 10 or more people are killed, 100 
or more affected, a state of emergency is declared or a 
call for international assistance is made. As we deliber- 
ately excluded much of the literature from low- and 
middle-income countries, percentages shown relate to 
EM-DAT records for high-income countries. This shows 
that the US and Canada are relatively over-represented 
in the literature, while continental Europe, Australasia 
and Japan are relatively under-represented. 

Themes addressed in each publication are shown in 
Table 4. Table 5 shows the thematic analysis of publica- 
tions in the event report, narrative review, systematic 



review, modelling, survey and RCT categories. These 
demonstrate a predominance of publications (over half) 
relating to emergency planning and emergency response, 
with relatively few addressing mitigation, with 219 of 
362 event reports describing the emergency response. 

Grey literature review 

192 documents were initially identified, of which 97 
were relevant. Of these, 52 were published by the Health 
Protection Agency. Table 3 shows the type of publication 
and type of event. The preponderance of publications re- 
lating to CBRN probably reflects the accessibility of Health 
Protection Agency reports. As in the academic literature, 
the majority of documents addressed emergency response, 
although there was significant discussion of organisa- 
tional issues and communications. 



Table 2 Country demographics of publications 




Number of 
publications 


%age 
publications 


Number of EMDAT entries 
1990-2011 n = 13378 


%age EMDAT entries 
from high income 
countries (n = 2777) 


Country 










UK 


47 


3.0 


86 


3.1 


US/Canada 


701 


45.4 


755 


27.2 


Europe (not UK) 


114 


7.4 


937 


33.7 


Japan 


45 


2.9 


149 


5.4 


Australasia 


26 


1.7 


359 


12.9 


Other high-income 


51 


3.3 


491 


1/./ 


Low/middle-income 


178 


11.5 


10601 




Multiple 


75 


4.9 






Unspecified 


308 


19.9 
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Table 3 Disaster and publication type 





Number academic %age academic 


Number of grey 


%age of grey 




publications 


publications 


documents 


documents 


Disaster type 










Natural eg earthquake 


339 


21.9 


16 


16.5 


CBRN 


135 


8.7 


44 


45.4 


Terrorism 


119 


/./ 


8 


8.2 


Outbreak/epi/pandemic 


89 


5.8 


/ 


7.2 


Industrial 


42 


2.7 


12 


12.3 


Transport 


35 


2.3 


2 


2.1 


Generic 


647 


41.9 


5 


5.2 


Multiple 


84 


5.4 


3 


3.1 


Other 


55 


3.5 






Publication type 










Commentary/editorial 


422 




9 


9.3 


Event report 


371 




74 


76.3 


Expert guidance 


122 




4 


4.1 


Survey 


118 








Educationalist 


75 




1 


1 


Modelling 


73 








Narrative review 


51 




1 


1 


Systematic review 


11 








RCT 


2 








Other 


300 




8 


8.2 


Discussion 




were excluded from the scope of this review. The prolif- 


This scoping review demonstrates that the published lit- 


eration of emerj 


;ency planning research in the US may 


erature relating to health 


emergency planning is dispro- 


represent the effect of widespread federal funding fol- 


portionately centred on North America; Australasia and 


lowing the 9/11 attacks. 




Europe have produced surprisingly little academic litera- 


To our knowledge this represents the first attempt to 


ture given the number of reported incidents in these 


scope the emerj 


jency planning literature 


relevant to a 


areas, although non-English publications from Europe 


UK setting. This scoping review used 


a deliberately 


Table 4 Thematic analysis of publications 








Theme 


Number of academic 


%age of academic 


Number of grey 


%age of grey 




publications 


publications* 


documents 


documents* 


Mitigation 


50 


3.1 


18 


18.6 


Hazard analysis 


124 


7.8 


27 


27.8 


Capability assessment 


167 


10.5 






Emergency planning 


391 


24.5 


30 


30.1 


Capability maintenance 


235 


14.7 


10 


10.3 


Emergency response 


557 


34.9 


53 


54.6 


Recovery 


177 


11.1 


27 


27.8 


Development plans 


91 


5.7 






Communications/mass media 


156 


9.8 


35 


36.1 


nformatics and intelligence 


183 


11.5 


26 


26.8 


Other organisational issues 


183 


11.5 


35 


36.1 



*Total exceeds 100% due to multiple themes in individual publications. 
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Table 5 Thematic analysis by publication type 





Event 
report 
(n = 362) 


Narrative 
review 
(n = 53) 


Systematic 
review 
(n = 11) 


Modelling 

(n = 73) 


Survey 
(n = 1 18) 


RCT 

(n = 2) 


Mitigation 


5 


2 




3 


1 




Hazard analysis 


/ 


3 




29 


8 




Capability assessment 


1 / 


6 


2 


9 


45 


1 


Emergency planning 


r / 
J/ 




A 

4 


1 o 






hi 1 it\/ m^intpn^inrp 

\— G L-'Gl LJI 1 1 Ly 1 1 la 1 1 I LCT 1 1 □ 1 


34 




] 




36 


1 


Emergency response 


219 


19 


6 


14 


22 


1 


Recovery 


49 


/ 


2 


2 


9 




Development plans 


16 


3 


1 




3 




Communications/ mass 
media 


33 


3 




3 


8 




Informatics/ intelligence 


28 


3 




21 


/ 




Other organisational issues 


32 


8 




1 


9 





Total in each column exceeds n due to multiple themes in individual publications. 



inclusive strategy and therefore our findings may over- 
estimate the extent of useable evidence. Equally, we did 
not assess the methodological quality of individual publi- 
cations beyond their stated design. It is unclear whether 
the traditional hierarchy of evidence applied to clinical 
research (privileging meta-analyses and randomised con- 
trolled trials) [12] is appropriate in this field. Except in 
very specific areas (for example brief psychological inter- 
ventions for survivors) randomised controlled trial 
designs are not practical. Narrative synthesis of observa- 
tional studies may be possible [13] but is currently ham- 
pered by the unevenness of collection. Repeated calls 
have been made to standardise major incident reporting 
and suggestions exist for the format this should take 
[14,15]. Even though multiple event reports exist there is 
little indication that this potential evidence base has 
been fully exploited through synthesis. We found it diffi- 
cult to identify a repository of emergency planning lit- 
erature that was easily and openly accessible. The two 
main repositories were the HPA's Chemical Incidents 
and Poisons Report database and the Emergency Plan- 
ning College library. Access to the Emergency Planning 
College library at Easingwold is controlled and their col- 
lection is not open to the public. 

We have identified a substantial amount of literature 
relating to emergency response. However, the validity and 
generalisability of the data is unclear, as is its amenability 
to synthesis and how this should inform policy and 
practice. We have also exposed the limited evidence base 
available to assist emergency planners and policymakers in 
the areas of mitigation and recovery. The development of 
an evidence base for emergency planning needs to take into 
account practical considerations regarding what type of evi- 
dence is most likely to be influential, as well as theoretical 
considerations of what constitutes valid evidence. The type 



and structure of evidence that would be of most value of 
emergency planners and policymakers has yet to be 
identified. Further research into stakeholder perceptions 
will be required to determine this. 

Conclusions 

A significant body of emergency planning literature exists. 
Nevertheless its usefulness to UK planners is constrained 
because it is dominated geographically by the US, themat- 
ically by emergency response topics and methodologically 
by event reports. 
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